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Automation  Architecture  for  Single 
Operator,  Multiple  UAV  Command 
and  Control 

M.L.  Cummings,1  S.  Bruni,  S.  Mercier,  and IBJ  Mitchell 

Abstract 

In  light  of  the  Office  of  the  Secretary  Defense’s  Roadmap  for 
unmanned  aircraft  systems  (UASs),  there  is  a  critical  need  for 
research  examining  human  interaction  with  heterogeneous 
unmanned  vehicles.  The  OSD  Roadmap  clearly  delineates  the 
need  to  investigate  the  “appropriate  conditions  and  requirements 
under  which  a  single  pilot  would  be  allowed  to  control  multiple 
airborne  UA  (unmanned  aircraft)  simultaneously”  Toward  this 
end,  in  this  paper,  we  provide  a  meta-analysis  of  research  studies 
across  unmanned  aerial  and  ground  vehicle  domains  that  investi¬ 
gated  single  operator  control  of  multiple  vehicles.  As  a  result,  a 
hierarchical  control  model  for  single  operator  control  of  multiple 
unmanned  vehicles  (UV)  is  proposed  that  demonstrates  those 
requirements  that  will  need  to  be  met  for  operator  cognitive  sup¬ 
port  of  multiple  UV  control,  with  an  emphasis  on  the  introduction 
of  higher  levels  of  autonomy.  The  challenge  in  achieving  effective 
management  of  multiple  UV  systems  in  the  future  is  not  only  to 
determine  whether  automation  can  be  used  to  improve  human 
and  system  performance,  but  how  and  to  what  degree  across  hier¬ 
archical  control  loops,  as  well  as  determining  the  types  of  decision 
support  that  will  be  needed  by  operators  given  the  high-workload 
environment.  We  address  when  and  how  increasing  levels  of  auto¬ 
mation  should  be  incorporated  in  multiple  UV  systems  and  dis- 
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cuss  the  impact  on  not  only  human  performance,  but  more 
importantly,  on  system  performance. 

Keywords:  multiple  unmanned  aerial  vehicles,  supervisory  con¬ 
trol,  operator  capacity 


Introduction 

Unmanned  aerial  vehicles  (UAVs)  are  quickly  becoming  ubiquitous 
in  military  command  and  control  operations.  With  reduced  radar 
signatures,  increased  endurance,  and  the  removal  of  humans  from 
immediate  threat,  unmanned  (also  known  as  uninhabited)  aerial 
vehicles  have  become  indispensable  assets  to  militarized  forces 
around  the  world,  as  proven  by  the  extensive  use  of  the  Shadow 
and  Predator  in  recent  conflicts.  Despite  the  absence  of  a  crew 
onboard  any  of  these  UAVs,  human  operators  are  still  needed  for 
supervisory  control. 

UAVs  require  human  guidance  to  varying  degrees  and  often 
through  several  operators,  which  is  what  essentially  defines  a  UAS 
(Unmanned  Aerial  System).  For  example,  the  Predator  and  Shadow 
each  require  a  crew  of  two  to  be  fully  operational.  Flowever,  with 
current  military  focus  on  streamlining  operations  and  reducing 
manning,  there  has  been  an  increasing  effort  to  design  systems  such 
that  the  current  many-to-one  ratio  of  operators  to  vehicles  can  be 
inverted.  In  light  of  the  Office  of  the  Secretary  of  Defense  Roadmap 
for  UASs  (2005),  there  is  a  critical  need  for  research  examining 
human  interaction  with  multiple  UASs,  which  has  also  recently 
been  identified  as  an  essential  need  by  the  Committee  on  Autono¬ 
mous  Vehicles  in  Support  of  Naval  Operations  (Naval  Studies 
Board  2005). 

In  response  to  this  need,  this  paper  first  defines  human  supervisory 
control  of  one  and  multiple  UAVs.  It  then  analyzes  past  literature  to 
examine  potential  trends  in  supervisory  control  research  of  multiple 
UAVs.  Specific  attention  is  paid  to  automation  strategies  for  deci- 
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sionmaking  and  action  as  levels  of  embedded  automation  signifi¬ 
cantly  differ  between  applications.  We  will  demonstrate  that  as 
autonomy  increases  across  hierarchical  control  loops,  the  number  of 
vehicles  a  single  operator  can  control  also  increases.  However,  as  we 
will  discuss,  increasing  system  autonomy  can  introduce  negative 
consequences  in  term  of  operator  situation  awareness  and  compla¬ 
cency.  The  focus  of  this  paper  is  primarily  on  human  supervisory 
control  of  multiple  UASs,  with  a  discussion  of  broader  command 
and  control  implications.  However,  the  reader  is  referred  to  (Alberts 
et  al.  1999;  Alberts  &  Hayes  2006;  Curts  &  Frizzell  2005;  Bolia  et  al. 
2006)  for  more  detailed  discussions  about  other  socio-technical 
implications  of  automation  and  network-centric  operations  in  com¬ 
mand  and  control  domains. 


Supervisory  Control  of  Multiple  UAVs 

The  move  from  platform-centric  warfare  to  Network  Centric  War¬ 
fare  (NCW)  represents  a  shift  in  the  role  of  humans  both  in  mission 
planning  and  actual  operation.  As  has  already  been  evidenced  in 
the  development  of  fly-by-wire,  highly  automated  aircraft,  and  mis¬ 
sile  systems  (such  as  Tomahawk  and  Patriot),  military  operators  are 
less  in  direct  manual  control  of  systems,  but  more  involved  in  the 
higher  levels  of  planning  and  decisionmaking  and  remote  opera¬ 
tions.  This  shift  in  control  from  lower  level  skill-based  behaviors  to 
higher  level  knowledge -based  behaviors  is  known  as  human  supervi¬ 
sory  control  (HSC).  HSC  is  the  process  by  which  a  human  operator 
intermittently  interacts  with  a  computer,  receiving  feedback  from 
and  providing  commands  to  a  controlled  process  or  task  environ¬ 
ment,  which  is  connected  to  that  computer  (Figure  1)  (Sheridan 
1992).  All  UAVs  in  the  DoD  inventory  operate  at  some  level  of 
supervisory  control  as  depicted  in  Figure  1 . 

Human  supervisory  control  in  UAV  operation  is  hierarchical,  as 
represented  in  Figure  2.  The  innermost  loop  of  Figure  2  represents 
the  basic  guidance  and  motion  control,  which  is  the  most  critical 
loop  that  must  obey  physical  laws  of  nature  such  as  aerodynamic 
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Figure  1.  Human  Supervisory  Control  (Sheridan  1992). 


Figure  2.  Hierarchical  Control  Loops  for  a  Single  UAV. 


constraints  for  UAVs.  In  this  loop,  operator  actions  are  focused  only 
on  the  short  term  and  local  control  (keeping  the  aircraft  in  stable 
flight),  and  generally  human  control  in  this  loop  requires  skill-based 
behaviors  that  rely  on  automaticity  (Rasmussen  1983). 

The  second  loop,  the  navigation  loop,  represents  the  actions  that 
some  agent,  whether  human  or  computer-driven,  must  execute  to 
meet  mission  constraints  such  as  routes  to  waypoints,  time  on  tar¬ 
gets,  and  avoidance  of  threat  areas  and  no-fly  zones.  The  outermost 
loop  represents  the  highest  levels  of  control,  that  of  mission  and  pay- 
load  management.  In  this  loop,  sensors  must  be  monitored  and 
decisions  made  based  on  the  incoming  information  to  meet  overall 
mission  requirements.  In  this  loop,  decisions  require  knowledge- 
based  reasoning  that  includes  judgment,  experience,  and  abstract 
reasoning  that  in  general  cannot  be  performed  by  automation. 

Finally,  the  system  health  and  status  monitoring  loop  in  Figure  2 
represents  the  continual  supervision  that  must  occur,  either  by  a 
human  or  automation  or  both,  to  ensure  that  all  systems  are  operat- 
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ing  within  normal  limits.  The  control  loop  line  is  dashed  as  it  repre¬ 
sents  a  highly  intermittent  loop  in  terms  of  the  human,  i.e.,  if  the 
human  is  engaged  in  another  task,  with  the  highest  priority  given  to 
the  innermost  loop,  health  and  status  monitoring  becomes  a  distant, 
secondary  task. 

From  the  human-in-the-loop  perspective,  if  the  inner  loops  fail,  then 
the  higher  (outer)  loops  will  also  fail.  The  dependency  of  higher 
loop  control  on  the  successful  control  of  the  lower  loops  drives 
human  limitations  in  control  of  a  single  and  especially  multiple 
UAVs.  If  humans  must  interact  in  the  guidance  and  motion  control 
loop  (hand  fly  a  UAV),  the  cost  is  high  because  this  effort  requires 
significant  cognitive  resources.  What  little  spare  mental  capacity  is 
available  must  be  divided  between  the  navigation  and  mission  man¬ 
agement  control  loops.  Violations  of  the  priority  scheme  repre¬ 
sented  in  Figure  2  have  led  to  serious  problems  exemplified  by 
numerous  Predator  crashes.  When  operators  become  cognitively 
saturated  or  do  not  correctly  allocate  their  cognitive  resources  to  the 
appropriate  control  loops  in  the  correct  priorities,  they  violate  the 
control  loops  constraints,  potentially  causing  catastrophic  failure. 

While  Figure  2  demonstrates  supervisory  control  at  the  single  vehi¬ 
cle  level,  Figure  3  represents  a  notional  system  architecture  that  will 
be  required  for  single  operator  control  of  multiple  UASs.  In  order  to 
achieve  this  futuristic  system,  operators  will  need  to  interact  with  an 
overall  mission  and  payload  manager  while  relegating  routine  navi¬ 
gation  and  motion  control  tasks  to  automation.  The  challenge  in 
achieving  effective  management  of  multiple  UAVs  in  the  future  is 
not  only  to  determine  if  automation  can  be  used  to  reduce  work¬ 
load,  but  how  and  to  what  degree  in  each  of  the  control  loops  in 
Figures  2  and  3,  as  well  as  what  kinds  of  decision  support  will  be 
needed  by  operators  given  the  high-workload  environment.  More¬ 
over,  the  depiction  of  a  single  mission  and  payload  management  sys¬ 
tem  for  multiple  vehicles  as  seen  in  Figure  3  is  not  just  specific  to 
UASs,  as  any  vehicle — manned  or  unmanned,  above  or  on  the 
ground,  or  under  water — can  be  a  node  under  a  common  mission 
management  system.  Thus  research  that  advances  single  operator, 


6  The  International  C2  Journal  |  Vol  1,  No  2 


multiple  UAV  command  and  control  capabilities  will  actually  set 
the  stage  for  the  implementation  of  NCW  concepts  in  general. 


System  Health  & 
Status  Monitoring 


Mission  & 
Payload 
Management 


Figure  3.  Hierarchical  Control  for  Multiple  Unmanned  Vehicles. 


Levels  of  Automation 

Increasing  the  autonomy  across  the  three  control  loops  discussed 
previously  (Figures  2  and  3)  is  the  critical  architecture  component 
for  allowing  one  or  a  small  team  of  operators  to  effectively  control 
multiple  UAVs.  By  increasing  UAS  autonomy,  operator  workload 
will  theoretically  be  reduced  as  it  could  reduce  the  number  of  tasks 
for  the  operator,  and  it  should  reduce  the  level  of  interaction  even  at 
the  highest  levels  of  control  in  Figures  2  and  3.  For  example,  those 
UAVs  that  are  flown  in  an  autopilot  mode  relieve  the  operator  from 
the  manual  flying  tasks  that  require  significant  cognitive  resources. 
This  frees  the  operator  to  perform  other  critical  tasks  like  mission 
planning  and  imagery  analysis. 

Higher  levels  of  automation  across  the  control  loops  depicted  in  Fig¬ 
ures  2  and  3  will  be  critical  in  achieving  the  single  operator,  multiple 
UAV  control  vision;  but  how,  when,  where,  and  what  level  of  auto¬ 
mation  should  be  introduced  are  still  difficult  problems.  While 
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workload  mitigation  can  occur  through  increasing  automation,  it 
can  inadvertently  cause  higher  workload  as  well  as  loss  of  situational 
awareness,  complacency,  and  skill  degradation  (Parasuraman  et  al. 
2000).  For  example,  some  UAV  researchers  have  found  that  inter¬ 
mediate  levels  of  management-by-consent  (automation  as  an  assis¬ 
tant  to  the  operator)  is  preferable  to  manual  or  more  fully 
automated  control  (Ruff  et  al.  2002).  Flowever,  management-by- 
consent  means  that  the  number  of  tasks  could  be  high  since  opera¬ 
tors  must  always  be  in  the  loop,  potentially  saturating  operators, 
especially  in  the  multiple  UAV  domain.  Moreover,  as  has  been 
shown  in  multiple  UAV  control  research,  operator  performance  can 
dramatically  decrease  under  management-by-consent  given 
increasing  workload  and  various  decision  aids  (Cummings  et  al. 
2007a;  Cummings  &  Mitchell  2006). 

Given  that  an  increasing  number  of  tasks  will  have  to  be  auto¬ 
mated  to  achieve  single  operator  control  of  multiple  UAVs,  the 
question  then  becomes  what  to  allocate  to  automation.  Previous 
research  has  demonstrated  that  in  the  scheduling  and  execution  of 
high-level  tasks,  of  multiple  UAVs,  management-by-exception  can 
improve  operator  performance  (Cummings  &  Mitchell  2006). 
Management-by-exception  occurs  when  automation  decides  to 
take  an  action  based  on  some  set  of  pre-determined  criteria,  and 
only  gives  operators  a  chance  to  veto  the  automation's  decision. 
While  this  control  scheme  can  be  effective  in  time-critical,  high- 
risk  domains  like  shutting  down  a  near-critical  reactor,  in  inten¬ 
tional,  highly  uncertain  domains  like  command  and  control,  it  can 
be  dangerous.  Under  this  control  scheme,  operators  are  more 
likely  to  exhibit  automation  bias,  a  decision  bias  that  occurs  when 
operators  become  over-reliant  on  the  automation  and  do  not 
check  to  ensure  automated  recommendations  are  correct  (Mosier 
&  Skitka  1996). 

Automation  bias  was  operationally  seen  in  the  2004  war  in  Iraq 
when  the  U.S.  Army’s  Patriot  missile  system,  operating  in  a  man- 
agement-by-exception  mode,  engaged  in  fratricide,  shooting  down 
a  British  Tornado  and  an  American  F/A-18,  killing  three.  The  sys- 
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tem  was  designed  to  operate  under  management-by-exception  and 
operators  were  given  approximately  15  seconds  to  veto  a  computer 
solution.  Unfortunately  the  displays  were  confusing  and  often  incor¬ 
rect,  and  operators  admittedly  lacked  training  in  the  highly  complex 
system  (32nd  Army  2003).  Given  the  laboratory  evidence  that  given 
an  unreliable  system,  humans  are  still  likely  to  approve  computer- 
generated  recommendations  (Cummings  2004),  it  is  not  surprising 
that  under  the  added  stress  of  combat,  Patriot  operators  did  not 
veto  the  computer’s  solution.  Automation  bias  is  a  significant  con¬ 
cern  for  command  and  control  systems  so  it  will  be  critical  to  ensure 
that  when  higher  levels  of  automation  are  used,  especially  at  the 
management-by-exception  level,  that  this  effect  is  minimized. 


A  Meta-Analysis  of  Previous  Multiple  UAV  Studies 

There  have  been  numerous  research  studies  published  that  have 
examined  various  aspects  of  multiple  UAV  control.  We  performed  a 
meta-analysis  across  those  studies  that  focused  either  explicitly  on 
operator  capacity  or  human  supervisory  control  aspects  of  multiple 
vehicle  control  in  order  to  determine  any  significant  trends  or  les¬ 
sons  learned,  particularly  in  regards  to  levels  of  automation  and  the 
control  loops  discussed  above. 


Case  Study  Summaries 

One  solution  investigated  by  Dixon  et  al.  to  reduce  UAV  operator 
workload  in  the  control  of  one  or  more  (small)  UAVs,  such  as  the 
Shadow,  consisted  of  adding  auditory  and  automation  aids  to  sup¬ 
port  the  potential  single  operator  (Dixon  et  al.  2005).  They  showed 
that  a  single  operator  could  theoretically  fully  control  a  single  UAV 
(both  navigation  and  payload)  if  appropriate  offloading  strategies 
were  provided.  For  example,  aural  alerts  improved  performance  in 
the  tasks  related  to  the  alerts,  but  not  others.  Conversely,  it  was  also 
shown  that  adding  automation  benefited  both  tasks  related  to  auto¬ 
mation  (e.g.,  navigation,  path  planning,  or  target  recognition)  as  well 
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as  non-related  tasks.  However,  their  results  demonstrate  that  human 
operators  may  be  limited  in  their  ability  to  control  multiple  vehicles 
that  need  navigation  and  payload  assistance,  especially  with  unreli¬ 
able  automation.  These  results  are  concordant  with  the  single-chan¬ 
nel  theory,  stating  that  humans  alone  cannot  perform  high-speed 
tasks  concurrently  (Welford  1952;  Broadbent  1958).  However, 
Dixon  et  al.  propose  that  reliable  automation  could  allow  a  single 
operator  to  fully  control  two  UAVs. 

Reliability  and  the  related  component  of  trust  is  a  significant  issue  in 
the  control  of  multiple  uninhabited  vehicles.  Ruff  et  al.  (2002)  found 
that  if  system  reliability  decreased  in  the  control  of  multiple  UAVs, 
trust  declined  with  increasing  numbers  of  vehicles  but  improved 
when  the  human  was  actively  involved  in  planning  and  executing 
decisions.  These  results  are  similar  to  those  found  by  Dixon  et  al. 
(2004)  in  that  systems  that  cause  distrust  reduce  operator  capacity. 

In  addition,  Ruffet  al.  (2002;  2004)  determined  that  higher  levels  of 
automation  actually  degraded  performance  when  operators 
attempted  to  control  up  to  four  UAVs.  Results  showed  that  manage  - 
ment-by-consent  (in  which  a  human  must  approve  an  automated 
solution  before  execution)  was  superior  to  management-by-excep- 
tion  (where  the  automation  gives  the  operator  a  period  of  time  to 
reject  the  solution).  Management-by-consent  appeared  to  provide 
the  best  situation  awareness  ratings,  the  best  performance  scores, 
and  the  most  trust  for  controlling  up  to  four  UAVs. 

Dunlap  et  al.  (2006)  also  subscribe  to  management-by-consent  in 
their  development  of  a  distributed  architecture  to  control  multiple 
unmanned  combat  aerial  vehicles  (UCAVs).  In  this  system,  a  UCAV 
plan  is  proposed  by  the  automation  and  the  operator  can  either 
accept  or  reject  the  plan  or  submit  an  alternative.  This  recommen¬ 
dation  can  include  both  target  assignments  and  routing.  While  they 
tested  four,  six,  and  eight  UCAVs  with  increasing  levels  of  environ¬ 
mental  complexity,  their  final  design  limited  the  UCAV  loadout  at 
four.  In  one  experiment,  they  noted  that  automation  bias  was  a 
prevalent  problem,  stating  the  operators  “had  become  attenuated  to 
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automatically  accepting  the  usually  correct  proposals  from  the 
UCAVs,”  which  resulted  in  an  increased  kill  rate  for  no-targets 
under  the  higher  levels  of  automation. 

In  terms  of  actually  predicting  how  many  UAVs  a  single  operator 
can  control,  there  are  only  a  few  studies  that  examine  this  ques¬ 
tion.  Cummings  and  Guerlain  (2007)  showed  that  operators  could 
experimentally  control  up  to  twelve  Tactical  Tomahawk  Land 
Attack  Missiles  (TLAM)  given  significant  missile  autonomy.  Oper¬ 
ators  only  had  to  interact  in  the  mission  management  loop  and  all 
other  loops  were  highly  automated.  In  a  UCAV  setting,  Cum¬ 
mings  et  al.  (in  press;  2007b)  demonstrated  that  the  number  of 
UCAVs  that  a  single  operator  can  control  is  not  just  a  function  of 
the  level  of  decision  support  automation,  but  also  the  operational 
tempo  and  demands.  Operators  under  low  workload  performed 
well  regardless  of  the  level  of  decision  support  but  under  high 
workload,  performance  degraded.  When  considering  operational 
and  workload  demands  for  a  suppression  of  enemy  air  defenses 
mission,  operator  capacity  was  estimated  at  five  UCAVs. 

In  a  demonstration  of  the  capabilities  of  a  single  operator  attempt¬ 
ing  to  control  multiple  Wide  Area  Search  Munitions  (WASMs), 
given  high  levels  of  autonomy  across  all  control  loops  in  Figure  3 
with  only  higher-order  goal  tasking  for  mission  management,  Lewis 
et  al.  posit  that  an  operator  can  effectively  control  up  to  eight 
WASMs  (2006).  The  assumption  is  that  the  automation  embedded 
in  the  vehicles  coordinates,  without  human  intervention,  specific 
tasks  such  as  target  detection,  choice  of  the  most  appropriate  mem¬ 
ber  to  execute  the  mission,  etc.,  which  are  capabilities  that  are  not 
yet  operational.  The  WASM  study  is  similar  to  the  Tactical  Toma¬ 
hawk  study  in  that  all  flight  control  and  navigation  functions  are 
allocated  to  the  automation  alone  and  the  human  intervenes  for 
very  high-level  goal  management. 

Thus  there  have  been  a  cross-section  of  studies  that  have  examined 
operator  performance  and  capacity  in  the  control  of  multiple  UAVs; 
however,  it  is  not  clear  how  any  meaningful  comparisons  can  be 
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made  across  the  different  domains  primarily  because  of  two  param¬ 
eters:  (1)  what  constitutes  control  and  (2)  what  level  of  automation 
was  used  to  aid  the  operators?  In  order  to  more  directly  compare 
these  studies,  the  following  section  will  discuss  the  scale  on  which 
comparisons  can  be  made. 


Level  of  Automation  Trends 

In  this  meta-analysis,  we  extracted  the  maximum  number  of  UAVs 
that  an  operator  effectively  controlled  in  each  study.  It  should  be 
noted  that  in  all  of  these  reported  studies,  the  control  occurred  in 
simulated  test  beds  of  nredium-to-high  fidelity.  We  identified  what 
we  interpreted  to  be  the  approximate  levels  of  automation  (LOAs) 
across  the  control  loop(s)  from  Figure  3.  While  numerous  levels  and 
scales  of  automation  and  autonomy  have  been  proposed  (Parasura- 
man  et  al.  2000;  Endsley  &  Kaber  1999;  Wickens  et  al.  1998;  Ends- 
ley  1995),  we  chose  the  ten-level  scale  originally  proposed  by 
Sheridan  and  Verplank  (1978)  (SV  -  LOA),  as  this  is  a  commonly 
referenced  taxonomy.  We  combined  some  categories  in  Table  1  to 
reflect  functional  similarities.  For  example,  levels  7-10  were  com¬ 
bined  since  the  human  can  take  no  action.  Recognizing  that  differ¬ 
ent  stages  of  information  processing  can  be  supported  by 
automation  (Parasuraman  et  al.  2000),  the  decision  and  action 
selection  stage  is  represented  in  our  assessment. 

In  Table  2,  which  presents  a  summary  of  these  findings,  the  num¬ 
bers  of  UAVs  potentially  controllable  by  a  single  human  operator 
are  referenced  along  with  estimated  levels  of  automation  for  each  of 
the  three  control  loops  (MC  =  Motion  Control  inner  loop;  N  = 
Navigation;  MM  =  Mission  Management  outer  loop.).  It  should  be 
emphasized  that  the  LOAs  selected  were  approximate  since  they 
were  both  subject  to  interpretation  and  assigned  post  hoc  from  stud¬ 
ies  not  originally  intended  to  answer  our  research  question.  In  addi¬ 
tion,  in  many  simulations,  the  LOA  was  not  fixed  and  we  have 
indicated  the  range  of  LOAs  in  these  cases.  In  this  comparison,  we 
also  included  an  air  traffic  control  (ATC)  study  since  it  embodies 
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many  of  the  same  principles  of  human  supervisory  control  that  are 
relevant  to  the  control  of  multiple  UAVs  (Hilburn  et  al.  1997).  Since 
air  traffic  controllers’  primary  focus  is  safe  navigation  of  aircraft, 
there  is  no  associated  mission  management  control  loop. 


Table  1.  Levels  of  Automation. 


sv- 

LOA 

Our 

LOA 

Automation  Description 

1 

I 

The  computer  offers  no  assistance;  human  must  take 
all  decisions  and  actions. 

2 

II 

The  computer  offers  a  complete  set  of  decision/ action 
alternatives. 

3 

III 

The  computer  offers  a  selection  of  decisions/ actions. 

4/5 

IV 

The  computer  suggests  one  alternative  and  executes 
that  suggestion  if  the  human  approves  (management  by 
consent). 

6 

V 

The  computer  suggests  one  alternative  and  allows  the 
human  a  restricted  time  to  veto  before  automatic  exe¬ 
cution  (management  by  exception). 

7/8/ 

9/10 

VI 

The  human  is  not  involved  in  the  decisionmaking  pro¬ 
cess;  the  computer  decides  and  executes  autonomously. 

Table  2  reveals  interesting  trends.  Without  explicitly  discussing  it  in 
their  respective  studies,  all  researchers  automated  the  inner  motion 
control  loop  as  depicted  in  Figures  2  and  3.  Thus  some  form  of 
autopilot  was  needed  to  relieve  operator  workload  and  free  cogni¬ 
tive  resources  for  higher  loop  control.  To  achieve  the  goal  of  one 
person  controlling  many  UAVs,  operators  should  only  monitor  the 
piloting/ maneuvering  of  the  vehicle,  not  do  it  themselves.  Flowever 
this  is  a  cultural  problem  more  than  it  is  a  technological  problem,  as 
this  technology  is  available  today  in  all  UAVs,  but  resisted  in  some 
communities,  i.e.,  some  organizations  still  insist  that  a  human  “fly” 
the  vehicle  instead  of  commanding  it  using  various  flight  profiles. 
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Table  2.  Multiple  UAV  Study  Comp: 

prison. 

Experiment 

LOA 

Max 

MC 

N 

MM 

UV# 

1 

Dixon  et  al.  (2005)  (baseline) 

VI 

I 

I 

1 

2 

Dixon  et  al.  (2005)  (autopilot) 

VI 

IV 

I 

2 

3 

Dixon  et  al.  (2005)  (auto-alert) 

VI 

I 

IV 

2 

4 

Ruffet  al.  (2002;  2004) 

VI 

IV-V 

IV 

4 

5 

Dunlap  (2006) 

VI 

IV 

IV 

4 

6 

Cummings  et  al.  (in  press;  2007b) 

VI 

IV 

III-IV 

5 

7 

Lewis  et  al.  (2006) 

VI 

VI 

IV-V 

8 

8 

Cummings  &  Guelain  (2007) 

VI 

VI 

IV 

12 

9 

Hilburn  et  al.  (1997)  (ATC  Study) 

VI 

V 

N/A 

11 

In  addition,  Figure  4a  demonstrates  a  general  increasing  trend  in 
the  number  of  vehicles  an  operator  can  control  as  a  function  of 
increasing  automation  in  the  navigation  control  loop  of  Figure  2. 
Thus  given  increasing  navigation  support  and  a  fully  autonomous 
flight  control  system,  operators  can  handle  more  UAVs  when  they 
do  not  have  to  attend  to  local  and  even  global  navigation  concerns. 
The  highest  operator  capacity  was  seen  in  the  Tomahawk  missile 
and  WASM  domains  because,  as  they  are  one-way  UAVs  traveling 
at  high  speeds,  there  is  little  time  and  fuel  for  human  interaction 
beyond  high-level  goal  direction. 

When  examining  the  mission  and  payload  management  control 
loop,  Figure  4b  demonstrates  the  ability  of  operators  to  control 
more  vehicles  as  they  are  provided  with  increasing  automated  deci¬ 
sion  support.  It  is  interesting  to  note  that  given  some  automated 
navigation  assistance  and  management-by-consent  automation  in 
the  mission  management  loop,  there  is  a  convergence  of  operator 
capacity  at  4-5  vehicles  per  operator.  The  next  remarkable  increase 
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in  operator  capacity  (8-12  vehicles)  is  not  seen  until  management- 
by-exception  is  introduced  in  the  mission  management  or  naviga¬ 
tion  loops.  These  increased  levels  of  automation  will  be  critical  for 
increased  operator  capacity  since,  as  previously  discussed,  if  opera¬ 
tors  are  required  to  attend  to  local  navigation  functions,  they  simply 
do  not  have  the  cognitive  resources  to  successfully  attend  to  all  of 
the  tasks  in  the  mission  and  payload  management  loop. 


(a)  Navigation  Loop 


L*v*l  4aitAtn«4«M 


(b)  Mission  Management  Loop 


Figure  4.  Max  Number  of  UAVs  vs.  LOAs  for  the  (a)  Navigation  Loop 

9 

and  (b)  Mission  Management  Loop.- 

One  important  consideration  not  evaluated  here  (primarily  due  to 
lack  of  any  experimental  evidence)  is  the  impact  of  heterogeneous 
UAV  control.  While  all  the  studies  included  in  this  review  were 
homogeneous  (except  for  the  ATC  study),  the  future  of  UAV  opera¬ 
tions  will  include  mixed  operations  such  that  potentially  different 
UAV  platforms  will  interact  in  the  same  airspace.  In  addition, 
another  important  variable  is  that  operators  never  manage  multiple 
vehicles  in  isolation  and  are  part  of  a  larger  team,  so  this  effort 
needs  to  be  extended  to  collaborative  teaming  environments. 

Our  purpose  in  relating  prior  multiple  UAV  human-in-the-loop 
studies  is  not  to  make  claims  about  a  specific  theoretical  maximum 
for  a  given  LOA,  but  rather  to  illustrate  the  possibility  that,  with 
increasing  automation  in  the  motion  control  and  navigation  loops 


2.  The  numbers  for  each  data  point  match  the  study  numbers  in  Table  2. 
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and  collaborative  decision  support  for  the  mission  management 
loop,  it  is  likely  that  operators  can  control  an  increasing  number  of 
UAVs.  Table  2  and  Figure  4  represent  approximate  generalizations 
and  both  automation  decision  support  and  subsequent  display 
design  can  significantly  alter  human  performance  (Cummings  et  al. 
2007a;  Smith  et  al.  1997). 


Linking  Operator  Capacity  Estimates  with  Performance  Parameters 

One  of  the  limitations  common  across  the  previously  reviewed  stud¬ 
ies  is  the  lack  of  measurable  performance  metrics.  In  general,  the 
performance  of  the  operators  was  deemed  acceptable  as  a  function 
of  expert  observation,  which  is  a  valid  method  for  performance 
assessment  (Endsley  &  Garland  2000),  but  is  not  generalizable 
across  domains  and  not  useful  for  predictions  as  it  is  essentially  a 
descriptive  operator  capacity  prediction.  Thus,  what  is  needed  is 
some  kind  of  performance  metric  that  captures  both  aspects  of 
human  and  system  performance,  which  indicates  an  objective  level 
of  goodness  and/or  satisficing  (Simon  et  al.  1986)  (i.e.,  a  “good 
enough”  solution  as  opposed  to  optimal.)  Indeed,  the  focus  on  key 
performance  parameters  (KPPs)  is  a  major  focus  for  the  Depart¬ 
ment  of  Defense,  particularly  in  terms  of  network-centric  command 
and  control  (Joint  Chiefs  2007). 

Research  is  currently  underway  to  address  this  disconnect  between 
operator  capacity  and  performance,  which  has  resulted  in  the  devel¬ 
opment  of  many  possible  KPPs.  A  recent  study  demonstrated  that 
the  number  of  UCAVs  that  a  single  operator  can  control  is  not  just 
a  function  of  the  level  of  decision  support  automation,  but  is  inextri¬ 
cably  tied  to  both  mission  complexity  and  overall  system  perfor¬ 
mance  (Cummings  et  al.  2007b).  Using  human  experimentation  in 
a  multiple  UCAV  simulation  test  bed  and  a  simulated  annealing 
(SA)  technique  for  heuristic-based  optimization,  operator  perfor¬ 
mance  was  predicted  to  be  significantly  degraded  beyond  approxi¬ 
mately  five  UCAVs,  with  an  optimal  bound  between  two  and  four 
vehicles  (Figure  5).  The  KPP  in  this  model  was  cost,  which  took  into 
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account  not  just  operational  costs  such  as  fuel,  but  also  the  cost  of 
missed  targets  and  cost  in  terms  of  mission  delays  introduced  by 
inefficient  human  interactions.  The  solid  curve  in  Figure  5  repre¬ 
sents  a  theoretically  perfect  human  operator,  and  the  dotted  line 
represents  more  realistic  human  performance  that  accounts  for 
delays  due  to  inefficient  decisionmaking,  communication  problems, 
cognitive  load,  etc.  Thus,  the  performance  of  the  system  (the  auto¬ 
mation  and  the  operator)  can  vary  as  a  function  of  the  operator,  but 
can  also  vary  due  to  the  operational  constraints  such  as  number  of 
targets,  operational  costs,  etc.  This  variation  is  why  it  is  important  to 
explicitly  link  system  performance  to  operator  capacity. 


Reliability  and  Trust 

Human  operator  interaction  and  performance  using  automated 
decision  support  agents  have  been  shown  to  be  significantly  influ¬ 
enced  by  system  reliability  and  resultant  (dis)trust,  i.e.,  the  more  reli¬ 
able  automation  is,  the  more  trust  users  have  and  the  more  likely 
they  are  to  use  the  automation  (Lee  &  Moray  1992;  Lee  &  Moray 
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1994;  Parasuraman  1993).  Conversely,  systems  with  poor  or  incon¬ 
sistent  reliability  will  often  be  disregarded  (Parasuraman  &  Riley 
1997;  Riley  1994).  This  is  especially  true  in  C2  supervisory  control 
systems  where  operators  are  multitasking  while  monitoring  various 
mission-critical  automated  systems.  High  reliability  may  lead  to 
overtrust,  automation  bias,  complacency,  and  loss  of  situation 
awareness.  Degraded  reliability  can  lead  to  undertrust,  misuse,  and 
disuse  of  automation  (Cummings  &  Mitchell  2006;  Parasuraman  & 
Riley  1997;  Cummings  2004;  Parasuraman  et  al.  1993). 

Of  the  studies  reviewed  here,  only  a  few  examined  the  impact  of 
trust  and  reliability.  Addressing  reliability,  Ruff  et  al.  (2002;  2004) 
concluded  that  performance  decreases  observed  in  low  reliability 
conditions  was,  in  part,  due  to  lower  trust  in  the  automated  deci¬ 
sion-aid  provided  to  the  human  operator.  This  was  especially  true  as 
the  number  of  vehicles  per  operator  increased.  Examining  trust, 
Dixon  and  Wickens  et  al.  (2005;  2004)  investigated  the  effect  of  per¬ 
fectly  reliable  versus  degraded  automation  on  performance  in 
multi-UAV  monitoring  tasks.  They  concluded  that  reliable  automa¬ 
tion  could  alleviate  task  interference  and  reduce  workload,  and 
allow  a  single  operator  to  control  several  UAVs.  However,  even  per¬ 
fectly  reliable  automation  could  not  prevent  a  decrease  in  perfor¬ 
mance  when  workload  increased. 


Broader  Command  and  Control  Implications 

While  the  focus  of  this  paper  is  primarily  on  human  supervisory 
control  of  multiple  UASs,  there  are  many  parallels  across  more 
general  command  and  control  settings,  particularly  for  network- 
centric  operations.  At  a  more  fundamental  level,  the  multiple  vehi¬ 
cle  supervisory  control  paradigm  represents  an  attention  alloca¬ 
tion  problem  that  requires  an  operator  to  determine  how  and 
when  to  allocate  limited  cognitive  resources  to  multiple,  often 
competing  tasks.  Thus  a  single  operator  controlling  multiple 
unmanned  vehicles  is  analogous  to  command  and  control  tasks  in 
time-pressured,  uncertain,  and  dynamic  settings  that  concurrently 
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compete  for  operator  attention.  For  example,  in  many  cases  there 
is  a  negative  impact  on  primary  task  performance  when  military 
personnel  attempt  to  attend  to  multiple  chat  streams,  particularly 
in  operations  centers  (Caterinicchia  2003;  Boiney  2005;  Fleacox 
2003),  indicating  that  operators  are  struggling  with  attention  allo¬ 
cation,  especially  in  task  prioritization. 

For  the  multiple  unmanned  vehicle  problem,  as  well  as  more  gen¬ 
eral  problems  like  chat  management,  operator  capacity  may  ulti¬ 
mately  be  limited  by  the  heterogeneity,  as  opposed  to  being  strictly 
limited  by  the  number  of  vehicles /tasks  under  control.  Research  is 
currently  underway  to  determine  the  impact  of  task  heterogeneity 
as  opposed  to  vehicle  and  payload  heterogeneity  (Kilgore  et  al. 
2007),  as  well  as  the  development  of  a  class  of  metrics  to  measure 
attention  allocation  efficiency  (Crandall  &  Cummings  in  press). 

The  move  towards  network-centric  operations  will  mean  that  oper¬ 
ators  must  deal  with  many  competing  sources  of  incoming  informa¬ 
tion,  which  could  easily  overload  their  cognitive  bandwidth.  Thus, 
research  investigating  operator  capacity  limits  both  from  a  vehicle 
and  task  heterogeneity  perspective  is  critical  not  just  for  multiple 
unmanned  vehicle  control,  but  also  for  those  command  and  control 
settings  that  require  operators  to  make  critical  decisions  with  multi¬ 
ple  sources  of  time-sensitive  information. 


Conclusion 

In  response  to  the  OSD  UAS  Roadmap  and  the  future  vision  of 
NCW,  a  number  of  defense  industries  are  now  investigating  and 
developing  possible  platforms  and  workstations  to  enable  single 
operator  control  of  multiple  UAVs.  For  example,  QinetiQ_  recently 
demonstrated  that  multiple  self-organizing  UAVs  can  be  effectively 
controlled  by  an  operator  flying  on  a  fast  jet.  Lockheed  Martin  has 
developed  a  prototype  control  station  that  not  only  allows  for  single 
operator  control  of  multiple  UAVs,  but  unmanned  ground  vehicles 
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as  well.  Raytheon’s  Universal  Control  System  is  advertised  as  pro¬ 
viding  single  operator  control  of  multiple  UASs,  simultaneously. 

Despite  the  aggressive  development  of  these  multiple  UAV/UAS 
control  technologies,  as  we  have  demonstrated  here,  the  research 
community  is  only  beginning  to  grasp  the  nuances  of  human  inter¬ 
action  in  the  cognitively  demanding  environment  of  multiple  vehi¬ 
cle  control.  While  this  meta-analysis  suggests  promising  trends — in 
that  with  increasing  automation  across  all  three  control  loops,  it  is 
likely  that  operators  can  control  an  increasing  number  of  UAVs — 
the  actual  number  will  depend  on  a  variety  of  other  factors  (system 
reliability,  communication  bandwidth,  mission  context,  complexity, 
operational  tempo,  operator  training,  etc.). 

In  addition,  as  previously  discussed,  one  serious  drawback  to  the 
insertion  of  high  levels  of  autonomy  in  the  command  and  control  of 
UAVs  is  the  strong  possibility  that  automation  bias  could  lead  to 
complacency  and  erroneous,  if  not  catastrophic,  outcomes.  This 
decision  bias  has  not  only  been  seen  in  many  experimental  com¬ 
mand  and  control  settings  (Cummings  2004;  Dunlap  2006),  but  also 
in  reality  in  the  recent  Gulf  War  conflict  (32nd  Army  2003).  Thus, 
this  possible  negative  outcome  of  increasing  autonomy  presents 
designers  of  both  the  technology  as  well  as  the  encompassing  orga¬ 
nizational  structures  with  a  challenge:  How  to  design  reliable  and 
trustworthy  automated  systems  that  reduce  operator  workload,  but 
also  keep  their  situation  awareness  sufficiently  high  so  that  they  can 
recognize  and  intervene  when  automation  fails?  It  remains  to  be 
seen  whether  the  systems  in  development  today  will  actually 
account  for  all  of  these  socio-technical  variables  instead  of  just 
focusing  on  the  technical  ones. 

The  critical  lesson  to  be  learned  from  this  meta-analysis  is  that  the 
success  of  any  UAS,  and  more  generally  multiple  command  and 
control  task  management,  is  not  just  contingent  on  high  levels  of 
autonomy,  but  more  linked  to  robust  system  automation  strategies 
that  account  for  human  operators’  cognitive  abilities,  both  positive 
and  negative.  Thus,  it  is  critical  that  developers  of  multiple  UAS 
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control  stations  today  recognize  and  design  for  the  complex  interac¬ 
tions  between  highly  automated  systems  and  the  need  to  support 
human  knowledge -based  reasoning. 


Acknowledgments 

This  research  is  partially  funded  by  the  Office  of  Naval  Research. 
The  views  and  conclusions  presented  in  this  paper  are  those  of  the 
authors  and  do  not  represent  an  official  opinion,  expressed  or 
implied,  of  ONR. 


References 

32nd  Army  Air  and  Missile  Defense  Command.  2003.  Patriot  missile 
defense  operations  during  Operation  Iraqi  Freedom.  Washington: 
U.S.  Army. 

Alberts,  D.S.,  and  R.E.  Hayes.  2006.  Understanding  command  and  control. 
Washington:  Command  and  Control  Research  Program. 

Alberts,  D.  s.JJ.  Garstka,  and  F.P  Stein.  1999 .  Network  centric  warfare. 
Washington:  Command  and  Control  Research  Program. 

Boiney,  L.  2005.  Team  decision  making  in  time-sensitive  environments. 
Paper  presented  at  10th  ICCRTS  in  McLean,  VA. 

Bolia,  R.S.,  M.A.  Vidulich,  and  W.T.  Nelson.  2006.  Unintended  conse¬ 
quences  of  the  network-centric  decision  making  model:  Considering 
the  human  operator.  Paper  presented  at  CCRTS  in  San  Diego,  CA. 

Broadbent,  D.E.  1958.  Perception  and  communication.  Oxford:  Pergamon. 

Cateiinicchia,  D.  2003.  DoD  chat  use  exploded  in  Iraq.  Federal  Computer  Week. 

Crandall,  J.W.,  and  M.L.  Cummings.  In  press.  Identifying  predictive  met¬ 
rics  for  supervisory  control  of  multiple  robots.  IEEE  Transactions  on 
Robotics:  Special  Issue  on  Human-Robot  Interaction. 


CUMMINGS  ET  AL.  |  Automation  Architecture  21 


Cummings,  M.L.  2004.  Automation  bias  in  intelligent  time  critical  deci¬ 
sion  support  systems.  Paper  presented  at  AIAA  3rd  Intelligent  Sys¬ 
tems  Conference  in  Chicago,  IL. 

Cummings,  M.L.,  A.S.  Brzezinski,  and  J.D.  Lee.  2007a.  The  impact  of 
intelligent  aiding  for  multiple  unmanned  aerial  vehicle  schedule  man¬ 
agement.  IEEE  Intelligent  Systems:  Special  Issue  on  Interacting  with  Autonomy 
22(2):  52~59. 

Cummings,  M.L.,  and  EJ.  Mitchell.  2006.  Automated  scheduling  decision 
support  for  supervisory  control  of  multiple  UAVs.  AIAA  Journal  of  Aero¬ 
space  Computing  Information,  and  Communication  3(6):  294-308. 

Cummings,  M.L.,  and  EJ.  Mitchell.  In  press.  Predicting  controller  capac¬ 
ity  in  remote  supervision  of  multiple  unmanned  vehicles.  IEEE  Trans¬ 
actions  on  Systems,  Man,  and  Cybernetics  -  Part  A:  Systems  and  Humans. 

Cummings,  M.L.,  and  S.  Gueriain.  2007.  Developing  operator  capacity 
estimates  for  supervisory  control  of  autonomous  vehicles.  Human  Fac¬ 
tors  49 : 1—15. 

Cummings,  M.L.,  C.E.  Nehme,  andj.  Crandall.  2007b.  Predicting  opera¬ 
tor  capacity  for  supervisory  control  of  multiple  UAVs.  In  Innovations  in 
intelligent  machines:  Studies  in  computational  intelligence  70,  ed.  J.S.  Chahl,  L. 
C.  Jain,  A.  Mizutani,  and  M.  Sato-Uic. 

Curts,  R.J.,  andJ.P.  Frizzell.  2005.  Implementing  network-centric  com¬ 
mand  and  control.  Paper  presented  at  10th  ICCRTS  in  Washington, 
DC. 

Dixon,  S.,  C.D.  Wickens,  and  D.  Chang.  2004.  Unmanned  aerial  vehicle 
flight  control:  False  alarms  versus  misses.  Paper  presented  at  Humans 
Factors  and  Ergonomics  Society  48th  Annual  Meeting  in  New 
Orleans,  FA. 

Dixon,  S.,  C.D.  Wickens,  and  D.  Chang.  2005.  Mission  control  of  multiple 
unmanned  aerial  vehicles:  A  workload  analysis.  Human  Factors  47: 
479-487. 


22  The  International  C2  Journal  |  Vol  1,  No  2 


Dunlap,  R.D.  2006.  The  evolution  of  a  distributed  command  and  control 
architecture  for  semi-autonomous  air  vehicle  operations.  Paper  pre¬ 
sented  at  Moving  Autonomy  Forward  Conference  in  Grantham,  UK. 

Endsley,  M.R.  1995.  Toward  a  theory  of  situation  awareness  in  dynamic 
systems.  Human  Factors  37:  32-64. 

Endsley,  M.R.,  and  D.B.  Kaber.  1999.  Level  of  automation  effects  on  per¬ 
formance,  situation  awareness  and  workload  in  a  dynamic  control 
task.  Ergonomics  42:  462-492. 

Endsley,  M.R.,  and  D.J.  Garland.  2000.  Situation  awareness  analysis  and  mea¬ 
surement.  Mahwah:  Lawrence  Erlbaum  Associates,  Inc. 

Heacox,  N.J.  2003.  Survey  of  chat  usage  in  the  fleet  results.  San  Diego: 
Pacific  Science  &  Engineering  Group. 

Hilburn,  B.,  PG.  Jorna,  E.A.  Byrne,  and  R.  Parasuraman.  1997.  The 
effect  of  adaptive  air  traffic  control  (ATC)  decision  aiding  on  control¬ 
ler  mental  workload.  In  Human-automation  interaction:  Research  and  prac¬ 
tice,  ed.  Mustapha  Mouloua  and  Jefferson  M.  Koonce,  84-91. 
Mahwah:  Lawrence  Erlbaum. 

Joint  Chiefs  of  Staff.  2007.  Chairman  of  the  Joint  Chiefs  of  Staff  instruc¬ 
tion  62 12.0 ID.  DoD. 

Kilgore,  R.M.,  K.A.  Flarper,  C.  Nehme,  and  M.L.  Cummings.  2007.  Mis¬ 
sion  planning  and  monitoring  for  heterogeneous  unmanned  vehicle 
teams:  A  human-centered  perspective.  Paper  presented  at  AIAA  Info- 
tech@Aerospace  Conference  in  Sonoma,  CA. 

Lee,  J.D.,  and  N.  Moray.  1992.  Trust,  control  strategies  and  allocation  of 
functions  in  human-machine  systems.  Ergonomics  35:  1234-1270. 

Lee,  J.D,  and  N.  Moray.  1994.  Trust,  self-confidence,  and  operators’ 

adaptation  to  automation.  International  Journal  of  Human-Computer  Stud¬ 
ies  AO:  153-184. 


CUMMINGS  ET  AL.  |  Automation  Architecture  23 


Lewis,  M.,  J.  Polvichai,  K.  Sycara,  and  P.  Scerri.  2006.  Scaling-up  human 
control  for  large  UAV  teams.  In  Human  factors  of  remotely  operated  vehicles , 
ed.  N.  Cooke,  H.  Pringle,  H.  Pedersen,  and  O.  Connor,  237-250. 
New  York:  Elsevier. 

Mosier,  K.L.,  and  L.J.  Skitka.  1996.  Human  decision  makers  and  auto¬ 
mated  decision  aids:  Made  for  each  other?  In  Automation  and  human  per¬ 
formance:  Theory  and  applications,  human  factors  in  transportation ,  ed.  R. 
Parasuraman  and  M.  Mouloua,  201-220.  Mahwah:  Lawrence 
Erlbaum  Associates,  Inc. 

Naval  Studies  Board.  2005.  Autonomous  vehicles  in  support  of  naval 
operations.  Washington:  National  Research  Council. 

Office  of  the  Secretary  of  Defense  (OSD).  2005.  Unmanned  aircraft  sys¬ 
tems  (UAS)  roadmap,  2005-2030.  Washington:  DoD. 

Parasuraman,  R.  1993.  Effects  of  adaptive  function  allocation  on  human 
performance.  In  Human  factors  and  advanced  aviation  technologies ,  ed.  D.J. 
Garland  and  J.A.  Wise,  147-157.  Daytona  Beach:  Embry-Riddle 
Aeronautical  University  Press. 

Parasuraman,  R.,  and  VA.  Riley.  1997.  Humans  and  automation:  Use, 
misuse,  disuse,  and  abuse.  Human  Factors  39:  230-253. 

Parasuraman,  R.,  R.T.  Molloy,  and  I.L.  Singh.  1993.  Performance  conse¬ 
quences  of  automation-induced  “complacency.”  International  Journal  of 
Aviation  Psychology  3. 

Parasuraman,  R.,  T.B.  Sheridan,  and  C.D.  Wickens.  2000.  A  model  for 
types  and  levels  of  human  interaction  with  automation.  IEEE  Transac¬ 
tions  on  Systems,  Man,  and  Cybernetics  —  Part  A:  Systems  and  Humans  30: 
286-297. 

Rasmussen,  J.  1983.  Skills,  rules,  and  knowledge:  Signals,  signs,  and  sym¬ 
bols,  and  other  distractions  in  human  performance  models.  IEEE 
Transactions  on  Systems,  Alan,  and  Cybernetics  13:  257~266. 


24  The  International  C2  Journal  |  Vol  1,  No  2 


Riley,  V  1994.  A  theory  of  operator  reliance  of  automation.  In  Human  per¬ 
formance  in  automated  systems:  Current  research  and  trends ,  ed.  M.  Mouloua 
and  R.  Parasuraman,  8-14.  Hillsdale:  Lawrence  Erlbaum  Associates. 

Ruff,  H.A.,  G.L.  Calhoun,  M.H.  Draper,  J.V  Fontejon,  and  B.J.  Guilfoos. 
2004.  Exploring  automation  issues  in  supervisory  control  of  multiple 
UAVs.  Paper  presented  at  2nd  Human  Performance,  Situation 
Awareness,  and  Automation  Conference  (HPSAA II)  in  Daytona 
Beach,  FL. 

Ruff,  H.A.,  S.  Narayanan,  and  M.H.  Draper.  2002.  Human  interaction 
with  levels  of  automation  and  decision-aid  fidelity  in  the  supervisory 
control  of  multiple  simulated  unmanned  air  vehicles.  Presence  1 1 :  335- 
351. 

Sheridan,  T.B.  1992.  Telerobotics,  automation  and  human  supervisory  control. 
Cambridge:  The  MIT  Press. 

Sheridan,  T.B.,  and  W.L.  Verplank.  1978.  Human  and  computer  control 
of  undersea  teleoperators.  In  Alan-machine  systems  laboratory  report.  Cam¬ 
bridge:  MIT. 

Simon,  H.A.,  R.  Hogarth,  C.R.  Piott,  H.  Raiffa,  K.A.  Schelling,  R. 

Thaier,  A.  Tversky,  and  S.  Winter.  1986.  Decision  making  and  prob¬ 
lem  solving.  Paper  presented  at  Research  Briefings  1986:  Report  of 
the  Research  Briefing  Panel  on  Decision  Making  and  Problem  Solv¬ 
ing  in  Washington,  DC. 

Smith,  PJ.,  E.  McCoy,  and  C.  Layton.  1997.  Brittleness  in  the  design  of 
cooperative  problem-solving  systems:  The  effects  on  user  perfor¬ 
mance.  IEEE  Transactions  on  Systems,  Alan,  and  Cybernetics  27:  360-  370. 

Welford,  A.T.  1952.  The  psychological  refractory  period  and  the  timing  of 
high-speed  performance:  A  review  and  a  theory.  British  Journal  of  Psy¬ 
chology  43:  2-19. 

Wickens,  C.D.,  S.E.  Gordon,  and  Y  Liu.  1998.  An  introduction  to  human  fac¬ 
tors  engineering.  New  York:  Longman. 


